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STEREOTACTIC SURGICAL PROCEDURE APPARATUS AND METHOD 

This invention was made wKh U.S. Government support under Grant No. 
DMC-88S78S4 awarded by the National Science Foundation. The U.S. Government has 
certain rights in the invention. 



Background and Summary of the Invention 

The present invention relates to an apparatus and method for planning and 
guiding insertion of an object along a linear trajectory into a body. More particularly, the 

10 present invention relates to an apparatus and method for coordinating two captured 

fluoroscope images to permit efifeaive three-dimensional planning of the trajectory using 
only two-dimensional images. 

Numerous medical interventions involve placing a needle, drill, screw, nail, 
wire or other device in the body. In some cases the angle and position of the device are 

15 both of critical importance, for example in the drilling of a hole for a screw along the axis 
of a spinal pedicle. In other cases, it is primarily the positioning of the end-point of the 
device which is important, for example in placing a biopsy needle into a suspected tumor. 
In still other cases, the objective is only to define a point rather than a line, for example in 
targeting a tumor for radiation therapy. Many other examples exist, especially in the field 

20 of orthopaedics. 

The present invention is also relevant to the development of percutaneous 
technique. Executing a linear trajeaory for the insertion of instrumentation into the body 
through the skin is more difficult than open surgical technique, but the reduced 
invasiveness and trauma of percutaneous placement makes it desirable. 

25 Fluoroscopy is frequently used by surgeons to assist medical procedures. 

Continuous fluoroscopy during a surgical procedure is undesirable because it exposes the 
surgeon's hands to radiation. Furthermore, regardless of whether intermittent or 
continuous fluoroscopy is used, the resuhing images are two-dimensional while insertion 
of the surreal instrument requires three-dimensional awareness by the surgeon. 

30 The apparatus and method of the present invention involve acquisition and 

storage of two separate fluoroscopic images of the body, taken fi-om two different angles. 
Typically, although not necessarily, these would be an anterior/posterior (A/P) image 
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taken front-to-back of the patient, and a sagittal image taken side-to-side. These two 
fluoroscopic images are displayed on two adjacent computer morators. The surgeon uses 
a trackball or other computer input device to specify on the monitors an insertion point 
and an insertion trajeaory. 

5 A mechanical positioning device is then used to position a guide through 

which the surgeon performs the insertion of the surgical instrument. The positioning 
device may either be an active computer controlled manipulator such as a robot, or it may 
be a manually adjusted mechanical device which is set numerically in accordance with an 
output from the computer. 

10 The apparatus and method of the present invention establish the projective 

geometric relationships relating each of two acquired fluoroscopic images to the 
three-dimensional workspace around and wthin the patient's body, despite essentially 
arbitrary positioning of the fluoroscope. The two images then become a coordinated pair, 
which permits three-dimensional planning that might otherwise be expected to require a 

15 computed tomography (CT) scan. 

WWle the acquisition and display of two approximately orthogonal images 
may be expected to present the surgeon with the greatest ability to plan in three 
dimensions, two images are not strictly necessary. It is possible to use a single captured 
image for some procedures, particularly if the surgeon has adjusted the beam axis of the 

2 0 fluoroscope into alignment with the intended trajectory. Furthermore, more than two 
images could also be acquired and coordinated, should that be advantageous 

Several other approaches to stereotactic or robotic surgery, planned on a 
computer screen displaying medical images, have been described by other workers, and 
wiU be listed below. Some background is given here before discussing prior art. The 

2 5 method and apparatus of the presem invention constitute a technique we call coordinated 

fluoroscopy. Coordinated fluoroscopy is a technique for REGISTRATION and for 
SURGICAL PLANNING. It allows registration based on the acquired fluoroscopic 
images themselves, without requiring any additional measuring devices. It aUows 
three-dimensional surgical planning based on fluoroscopic views from two angles, without 

3 0 requiring three-dimensional imaging such as computed tomography (CT). and without 

requiring that the two fluoroscopic images be acquired from orthogonal fluoroscope 
poses. 
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REGISTRATION 

Registration is a key step in any image*guided surgical system. 
Registration is the determination of the correspondence between points of the image upon 
5 which a surgical plan is prepared, and points of the workspace in the vicinity of (and 
within) the patient. If a numerically controlled tool (whether robotic or manual) is to be 
used, the coordinate system of that device must also be brought into registry urith the 
image. 

It is common to accomplish registration v^th the help of a global 

10 positioning device, usually optical, which can measure the three-dimensional coordinates 
of markers placed anywhere over a large volume of space. Coordinated fluoroscopy 
avoids the necessity for this expensive and inconvenient device, instead deriving 
registration directly from the acquired fluoroscopic images themselves. Coordinated 
fluoroscopy uses a "registration artifact** which is held in a fixed position relative to the 

15 patient while one or more fluoroscopic images are acquired from different angles (poses). 
There is no need to constrain the fluoroscope poses at which these various images are 
acquired, for instance to require that they be orthogonal, nor is there a need to instrument 
the fluoroscope so that the pose angles can be measured. Instead, pose information is 
extracted after-the-fact from the images. It is a substantial benefit of the present invention 

2 0 that surgeons can acquire fluoroscopic images using fluoroscope poses of their own 
choosing, as they are accustomed. 

The registration artifact contains a plurality of features (fiducials) which 
are designed to be easily identifiable on a fluoroscopic image. The embodiment described 
here uses eight small steel spheres embedded in a radiolucent matrix. The positions of 

25 these fiducials are known relative to a coordinate system fixed in the artifact, either by 
design or by measurement. 

From the two-dimensional locations of the projections of these fiducials in 
a fluoroscopic image, we can determine the geometric projections that cany a general 
three dimensional point anywhere in the vicinity of the artifact into a projected point on 

30 the image. This establishes registration between image and workspace. Several images 
can each be registered relative to the same registration artifact, thus also bringing all the 
images into registry with one another. 
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Identiiication of the geometric projections, as discussed above, would not 
be possible with raw fluoroscope images, which are highly nonlinear and distorted. It is 
necessary first to map and compensate for these distortions. It is useful to be aware of 
the necessity of distortion compensation when comparing the present invention to prior 
5 art. 



SURGICAL PLANNING 

Surgical planning is also a key step in image-guided surgery. Planning of 
three-dimensional surgical procedures might be expected to be done on a 
10 three-dimensional dataset, such as can be reconstructed from computed tomography (CT) 
data. However, surgeons are accustomed to planning on two-dimensional images: 
radiographs or fluoroscopic images. Indeed even when CT data is available, planning is 
usually done on individual two-dimensional CT "slices" rather than on a three-dimensional 
reconstruction. 

15 The coordinates of the endpoints of a line segment representing an 

intended screw, biopsy needle, or drilled hole are of course three-dimensional, as are the 
coordinates of a single point within the body marking the present location of a tumor or a 
fragment of shrapnel. In surgical planning such points can be specified on a 
two-dimensional image, or on each of several two-dimensional images. Each such 

20 two-dimensional image is a projection of the same three-dimensional space 

It is necessary to convert the two-dimensional coordinates of specified 
points on each of several images into a three-dimensional coordinate which can be used to 
guide a tool along a desired trajectory or to a desired point within the body. To do so 
one must have knowledge of the geometric relationship of the projections that created the 

25 images. 

In the absence of such geometric knowledge a point specified on one 
image and a point independently specified on another image may in fact not correspond to 
any single point witMn the body. This is so because a point specified on a 
two-dimensional image is the projection of a LINE in space. The implied point in 
3 0 three-dimensions is the intersection of two such lines, one implied by the point specified 
on each image. Two such lines created independently may be skew, intersecting nowhere. 
Similarly, line segments for an intended procedure can not be chosen independently on 
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two images, otherwise they will in general not correspond to a well-defined 
three-dimensional line segment. 

In coordinated fluoroscopy, the geometric projections that relate the two 
images to a single three-dimensional coordinate system are established before planning 
commences. The points chosen by the surgeon on two (or more) images can therefore be 
constrained by the software such that they DO correspond to a well-defined point in 
three-dimensions. In practice, as a surgeon adjusts an intended point or line segment on 
one image, the point or line segment displayed on the other image(s) continuously updates 
and adjusts as well. One cannot draw "arbitrary" points or line segments independently on 
the images; the software only allows one to draw points or line segments that correspond 
to a well-defined point or line segment in three-dimensions. 

The benefits of planning on geometrically coordinated images as described 
above are threefold: 

1) Once the surgeon has selected a point or a line segment on two images, 
the three-dimensional point or line segment to which the selections correspond is fully 
defined and ready to be executed. 

2) An axial view such as could be attained from a CT slice is generally 
unattainable fluoroscopically. The angle that is most easily visualized in axial view, 
known as the transverse angle, is therefore difficult to select or execute under 
fluoroscopy. In coordinated fluoroscopy the transverse angle is implicitly specified by the 
surgeon by selecting line segments on two images. This may assist the surgeon in 
visualizing and planning the transverse angle for a procedure. 

3) In conventional fluoroscopy, image dilation due to beam divergence is 
of unknown extent, making accurate measurement of anatomic disunces difficuh. In 
coordinated fluoroscopy the actual in-situ length of an intended line segment can be 
determined by the software. This is useful for selecting appropriate screw length, as well 
as for other purposes. 

BACKGROUND 

Lavalle et al. in Grenoble, France have developed a system for spinal 
surgery which uses computed tomography as an image source. The CT data is assembled 
into a three-dimensional data set which can then be resliced at mil on orthogonal planes. 
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Surgical planning proceeds on three mutually orthogonal planes simultaneously. 
Registration is performed by using an optical tracking device to digitize arbitrary surface 
points of the vertebrae, and matches those surface points to the CT data set. 

Nolte ct al. in Bern, Switzerland have developed a very similar spinal 
5 system to Lavalle et al. Regisuation differs in that the optical tracking device is used to 
digitize specific anatomic landmarks rather than general surface contours. The features 
are then pointed out manually in CT data, allowing a match to be made. 

P. Finlay in High Wycombie, England has developed a fluoroscopic system 
for head-of-femur (Kp) fractures. Accuracy requirements in this procedure are not very 

10 great, so fluoroscope distortion compensation is not needed. Its absence also precludes 
identification of the geometric projections from images as is done in the present invention. 
Instead, the two fluoroscope poses are required to be orthogonal and the C-arm must not 
be moved along the floor in between the two images. Registration is accomplished by 
noting various features of a surgical tool which appears in the images, and by highlighting 

15 a marker wire which also appears in the field of view of the fluoroscope. 

Potamianos et al. in London. England have developed a system for kidney 
biopsy and similar sofl-tissue procedures. It incorporates a digitizing mechanical arm to 
which a biopsy needle is attached, and which can be moved about manually by the 
surgeon. Surgical planning per se is absent; instead a line segment representing the 

20 present position of needle is displayed superimposed upon captured (static) fluoroscope 
images, as the needle is moved manually near and within the patient. 

Phillips et al. in Hull, England have developed a system for orthopaedic 
procedures. It uses a optical tracking device as well as a fluoroscope. Registration is 
accomplished by instrumenting the fluoroscope with light emitting diodes and tracking 

25 them with the optical tracker. Surgical planning software is specific to the surreal 

procedure, and tends to offer medical opinion rather than just display a trajectory as in the 
present invention. For intramedullary nail placement, for instance, the surgeon outlines 
target holes in an intramedullary prosthetic, and software calculates a trajectory through 
them. 

30 U.S. Patent 4,750,487 (Zanetti) describes a stereotactic frame which 

overlays a patient. A single anterior/posterior fluorograph is then acquired, in which a 
crosshairs affixed to the frame is visible. By measuring the displacement of the crosshairs 
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from the desired target, a motion of the frame can be accomplished which brings the two 
into alignment. This invention does not facilitate three-dimensional stereotaxy as does the 
present invention. 

U.S. Patent 5,078.140 (Kwoh) describes a stereotactic and robotic system 
5 for neurosurgery. It uses CT images. 



ASPECTS OF THE INVENTION 

According to the present invention, a method is provided for planning a 
stereotactic surgical procedure for a linear trajectory insertion of surgical instrumentation 

10 into a body using a fluoroscope for generating images of the body. The method includes 
placing adjacent to the body a registration artifact containing a plurality of fiducials; 
displaying on a computer monitor an image of the patient's body and the registration 
artifact; receiving a user or automatic algorithmic input to identify two-dimensional 
coordinates of the fiducials of the registration artifact displayed on the first monitor; and 

15 registering the image by creating a geometric model having parameters, 

said model projecting three-dimensional coordinates into image points, and numerically 
optimizing the parameters of the geometric model such that the projections of the known 
three-dimensional coordinates of the fiducials best fit the identified two-dimensional 
coordinates in the image. 

2 0 The method further includes displaying on a second computer monitor a 

second image, taken of the patient's body and the registration artifact but from an angle 
different from that of the first image, and receiving a user or automatic algorithmic input 
to identify two-dimensional coordinates of the fiducials displayed on the second computer 
monitor; and registering the second image by creating a geometric model having 

25 parameters, said model projecting three-dimensional coordinates into image points, and 
numerically optimizing the parameters of the geometric model such that the projections of 
the known three-dimensional coordinates of the fiducials best fit the identified 
two-dimensional coordinates in the second image. 

The method, whether one or two images have been acquired, further 

30 includes the step of receiving a user input to select on a computer monitor an entry point 
for a surreal instrument. In the case of two images, also receiving a user input to select 
on a computer monitor the position, length, and angles of a virtual guidewire representing 
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the trajcaory for the surgical instrument; and drawing a segment, to be known as a 
PROJECTED GUIDEWIRE, on the imagc(s). When there are two images, the projected 
guidewires are constrained to correspond geometrically to the same three-dimensional 
segment in space, to be known as the VIRTUAL GUIDEWIRE. 

The method fiirther includes recdving a user input to move dther end of a 
projected guidewire, by revising the \artual guidewire of which the projeaed guidewire(s) 
are projections, and by redrawing the projected guidewires in correspondence with the 
revised virtual guidewire. 

The method further includes receiving a user input to change the length of 
the virtual guidewire, and redrawing the projected guidewire(5) in correspondence vnth 
the revised virtual guidewire. A special case is that the length is zero, so that ^at is 
planned is a virtual targetpoint rather than a virtual guidewire. 

The method further includes receiving a user input to change the sagittal, 
transverse, or coronal ang]e(s) of the virtual guidewire, updating the orientation of the 
virtual guidewire based on the new angles, and redrawing the projected guidewire(s) in 
correspondence with the revised virtual guidewire. 

The method fiuther includes producing an output to adjust the coordinates 
of a tool giude such that the projection of the axis of the guide in an inuige is brought into 
correspondence with the entry point displayed on the computer monitor. 

The method further includes producing an output to adjust the coordinates 
of a tool guide such that it is brought into correspondence with the virtual guidewire; or 
produdng an output to adjust the coordinates of a tool guide such that the position of the 
guide along its axis is offset by a preselected distance from one endpoint of the ^rtual 
guidewire, in order to control the location within the body of the surgical instrument to be 
inserted. ' 

The method further includes transmitting said coordiimtes to a robot or 
other automatic mechanical device, or displaying said coordinates such that human 
operator may manually adjust a mechanical device. 

Brief Descri^fti^n pf the Drawintts 

The detailed description particularly refers to the accompanying figures in 

wUch: 
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Fig. 1 is a diagranunatic illustration of the stereotactic surgical apparatus 
of the present invention for coordinating images from a fluoroscope, planning a linear 
trajectory medical intervention, and controlling a robot to control the linear trajectory 
medical intervention; 

Fig. 2 is a perspective view of a registration artifact and tool guide of the 
present invention; 

Fig. 3a is a sample screen display of the user interface which includes an 
anterior/posterior (A/P) taken by the fluoroscope and displayed on a first computer 
monitor along with a number of the buttons and entry fields necessary to run the program; 

Fig. 3b is a sample screen display which includes a sagittal image taken by 
the fluoroscope and displayed on a second computer moritor along with a number of the 
buttons and entry fields necessary to run the program; 

Fig. 3c is a flow chart of the steps performed by the computer during a 
main program loop; 

Fig. 4 is a flow chart illustrating the steps performed by the computer to 
acquire an A/P image firom the fluoroscope; 

Fig. S is a flow chart illustrating the steps performed by the computer to 
acquire a sagittal image fi-om the fluoroscope; 

Fig. 6 is a flow chart illustrating the steps performed by the computer and 
the user to selea or identify A/P fiducials fi-om the A/P image displayed in Fig. 3a; 

Fig. 7 is a flow chart of the steps perfonned by the computer and the user 
to select or identify sagittal fiducials displayed on the sagittal image of Fig. 3b; 

Fig. 8 is a flow chart illustrating the steps performed by the computer to 
register the A/P image; 

Fig. 9 is a flow chart illustrating the steps performed by the computer to 
register the sagittal image; 

Fig. 10 is a flow chart illustrating the steps performed by the computer for 
changing a transverse angle of the virtual guidewire; 

Fig. 1 1 is a flow chart illustrating the steps performed by the computer to 
change the length of the virtual guidewire used in the stereotactic surgical procedure; 

Fig, 12 is a flow chart illustrating the steps performed by the computer to 
change a sagittal angle of the virtual guidewire; 
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Fig. 13 is a flow chart Slustrating the steps performed by the computer to 
change the approach angle of the robot; 

Fig. 14 is a flow chart illustrating the steps performed by the computer to 
move the robot illustrated in Fig. 1 to the planned position and orientation; 
5 Fig. 15 is a flow chart illustrating the steps performed by the computer to 

move the end efifeaor of the robot along the axis of the tool guide; 

Fig. 16 is a flow chart illustrating the steps performed by the computer 
when the computer receives a user input based on a cursor in the A/P image area of Fig. 
3a; 

10 Fig. 17 is a flow chart illustrating the steps performed by the computer 

when the computer receives a user input based on a cursor in the sagittal image area in 
Fig. 3b; and 

Fig. 18 is a flow chart illustrating the steps performed by the computer 
when the computer receives a user input based on a cursor in the robot control areas of 
15 Figs. 3a-b. 

Detailed Description of Drawings 

Referring now to the drawings^ Fig. 1 illustrates the stereotaaic system 10 

for linear trajectory medical interventions using calibrated and coordinated fluoroscopy. 
20 The apparatus and method of the present invention is designed to utilize images firom a 

fluoroscope 12 such as a standard C*arm which generates fluoroscopic or x*ray images of 

a body on a surgical table 14. The imaging arm 16 is moveable so that both 

anterior/posterior (A/P) and sagittal or side images of the body can be taken. 

A robot 18 is situated adjacent the surgical table 14. Illustratively, the 
25 robot is a PUMA.S60 robot. The robot 18 includes a movable arm assembly 20 ha>dng 

an end flange 22. An alignment or re^stration artifact 24 is coupled to the end flange 22 

of robot 18. 

The registration artifact 24 is best illustrated in Fig. 2. The artifact 24 is 
X-ray and visually transparent with the exception of 8 opaque spheres or fiducials 26, and 
30 an aperture 30 to hold a tool guide 28 through the artifact 24. Initially, the artifact 24 is 
positioned roughly over the area of interest of body 32 and within the field of view of the 
fluoroscope 16. Therefore, the fiducials 26 show up as distinct dots on the A/P and 
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sagittal images as discussed below. The shape of the artifact is designed so that the image 
dots from the fiducials 26 will not over shadow each other and is sensitive to any angular 
deviations. The robot arm 20 can adjust the anifact 24 in three-dimensions about X-axis 
34, Y-axis 36. or Z-axis 38 illustrated in Fig. 1. 

The coordinated fluoroscopic control system of the present invention is 
controlled by computer 40, which includes a microprocessor 42. and internal RAM 44, 
and a hard disk drive 46. Computer 40 is coupled to two separate graphics monitors 48 
and 50. The first graphics monitor 48 displays a sagittal image taken by the C-arm 12. 
The second monitor 50 displays an A/P image taken by the C-arm 12, Computer 40 
further includes a serial communication port 52 which is coupled to a controller 53 of 
robot 18. Computer 40 is also coupled to C-arm 12 for receiving the images from the 
C-anm 12 through an image acquisition card 54. Computer 40 is also coupled to an input 
device 56 which is illustratively a keyboard having a track ball input control 58. Track 
ball input 58 controls a cursor on both monitor 48, SO. 

The displays on monitors 48 and 50 are illustrated in Figs. 3a and 3b. 
Referring now to Fig. 3b, the sagittal image is displayed in area 62 on monitor 48. All 
eight iiducials 26 should appear in the sagittal image area 62, If not. the artifact 24 or the 
C-arm 12 should be adjusted. As discussed in detailed below, computer 40 displays a top 
entry point 64 and a bottom point 66 of a projected guidcwire 68. The projeaed 
guidewire 68 is a line segment which is displayed on the sagittal image area representing 
the position of the instrumentation to be inserted during the stereotactic surgical 
procedure. A line of sight 70 is also displayed in the sagittal image area 62. 

Various user option buttons are displayed on monitor 48. The surgeon or 
operator can access these options by moving the cursor to the buttons and cliddng or by 
selerting the appropriate fiinction keys (Fl. F2, etc.) on the keyboard. The option 
buttons displayed on monitor 48 include button 72 (function F2) for acquiring the sagittal 
image, button 74 (F4) for selecting sagittal fiducials. and button 76 (F6) for registering 
the sagittal image. In addition, button 78 (FIO) is provided for setting the sagittal angle, 
button 80 (F8) is provided for setting the screw length, and button 82 (F12) is provided 
for moving the robot along an axis of the tool guide. Finally, the display screen includes a 
robot control area 84. The operator can move the cursor and click in the robot control 
area 84 to control robot 1 8 as discussed below. 
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Refening to Fig. 3a, the A/P image displayed on the display screen of 
monitor SO is illustrated. The A/P image is displayed in area 86 of the screen. Again, all 
eight fiducials 26 should appear within the A/P image area 86. The top insertion point of 
the virtual guidewire is illustrated at location 88, and the bottom point is located at 
5 location 90. The projection of the guidewire onto the A/P image is illustrated by line 
segment 92. 

Computer 40 also displays various option buttons on monitor SO. Button 
94 (Fl) is provided for acquiring the A/P image. Bunon 96 (F3) is provided for selecting 
the A/P fiducials. Button 98 (FS) is provided for registering the AP image. Button 100 

10 (F7) is provided for setting a transverse angle of the virtual guidewire, and button 102 
(F9) is proNaded for setting an approach angle for the robot. Button 104 (Fl 1) is 
provided for moving the robot. Computer 40 also displays a robot control area 84. The 
operator can move the cursor and click in the robot control area 84 to control robot 1 8 as 
discussed in detail below. 

15 The present invention allows the surgeon to select the point of entry for 

the surgical instalment by moving the top point of the projected guidewire 88 in the A/P 
image area 86. The operator can also adjust the bottom point of the projected guidewire 
90 to specify the transverse and sagittal angle. In addition, the operator can adjust the top 
point of the projected guidewire 64 to specify the position on the line of sight and bottom 

20 point of the projected guidewire 66 to specify the sa^ttal and transverse angle in the 
sagittal image area 62. Therefore, the surgeon can select the desired position and 
orientation of the surgical instrument into the body. 

The computer 40 is programmed with software to correct spatial 
distortions from the optics of the fluoroscope 12. The system of the present invention 

25 permits effective three-dimen^onal planning of the stereotactic surgical procedure using 
only a pair of two dimensional fluorographic images displayed on the adjacent monitors 
48 and SO. It is not required to use a CT slice in order to fully specify the location of the 
surgical instrument. The computer 40 establishes the direct geometric relationship 
between the A/P and sagittal images, despite image distortions and the essentially random 

30 or free-hand positioning of the C-arm 12, to establish the A/P and sagittal images. The 
improved system of the present invention can establish this exact geometric relationship 
within sub-miUimeter accuracy. 
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Once the sagittal and A/P images are registered, points or lines chosen by 
the surgeon on one of the A/P image or the sagittal image are immediately displayed by 
computer 40 as corresponding projections on the other image. Therefore, using the 
sagittal image on monitor 48 and the A/P image on monitor SO, the surgeon can 
5 stereotactically plan the linear trajectory without the requirement of CT scan slice. 
Accordingly, the procedure of the present invention can be performed without the very 
e)q)ensive CT scan devices which can cost in excess of SI million. 

Details of the operation of the software for controlling the system of the 
present invention are illustrated in Figs. 3c*18. 
10 All of the notations, subscripts and mathematical formulae, equations, and 

explanations are included in the attached Appendix. Throughout the flow charts 
described Figs. 4-1 8, reference will be made to the Appendix and to the numbered 
Sections [I] through [IS] set forth in the Appendix. 

The main program begins at block 1 10 of Fig. 3c. Computer 40 creates a 
15 parent window at block 1 12 and then draws buttons on a main window as illustrated at 
block 114. Computer 40 then creates a sagittal child window on morutor 48 as illustrated 
at block 1 16. Computer 40 also creates an A/P child window on monitor SO as 
illustrated at block 118. Computer 40 then determines whether a button or key has been 
pressed at block 120. If not, computer 20 waits as illustrated at block 122 and then 

2 0 returns to block 120 to wait for a button or key to be pressed. 

If a button or key was pressed at block 120, computer 40 determines 
whether the Acquire A/P Image button 94 or the Fl key was pressed at block 124. If so, 
computer 40 advances to block 166 of Fig. 4. If not, computer 40 determines whether the 
Acquire Sagittal Image bunon 94 or the F2 key was pressed at block 126. If so, the 
25 computer 40 advances to block 200 of Fig. S. If not, computer 40 determines whether the 
Select A/P Fiducial button 96 or the F3 key was pressed at block 128. If so, computer 40 
advances to block 234 of Fig. 6. If button 96 or the F3 key was not pressed at block 128, 
computer 40 determines whether the Select Sagittal Fiducial button 74 or the F4 key was 
selected as illustrated at block 130. If so, computer 40 advances to block 276 of Fig. 7. If 

3 0 not, computer 40 advances to block 132. 

In block 132, computer 40 determines whether the Register A/P Image 
button 98 or the F5 key was pressed. If so, computer 40 advances to block 324 of Fig, 8. 
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if not, computer 40 detennines whether the Register Sagittal Image button 76 or the F6 
was pressed as illustrated at blodc 134. If so, computer 40 advances to block 3S0 of Fig. 
9. If not, computer 40 advances to block 136. 

From block 136, computer 40 determines whether the Transverse Angle 
5 button 100 or the F7 key was pressed as illustrated at block 138. If so, computer 40 
advances to block 376 of Fig. 10. If not, computer 40 detennines whether the screw 
Length button 80 or F8 key was pressed as illustrated at block 140. If so, computer 40 
advances to block 388 of Fig. 1 1 . If not. computer 40 determines whether the Sagittal 
Angle button 78 or the FIO key was pressed as illustrated at block 142. If so, computer 
10 40 advances to block 400 of Fig. 12. If not, computer 40 determines whether the 

Approach Angle button 102 or the F9 key was pressed as illustrated at block 144. If so, 
computer 40 advances to block 412 of Fig. 13. If not, computer 40 advances to block 
146. 

In block 146, computer 40 determines whether the Move Robot button 

15 104 or the FII key was pressed. If so, computer 40 advances to block 422 of Fig. 14. If 
not, computer 40 determines whether the Move Robot Along Axis button 82 or the F12 
key was pressed as illustrated at block 148. If so, computer 40 advances to block 452 of 
Fig. IS. If not, computer 40 determines whether the A/P Image area of monitor SO has 
been selected by clicking when the cursor is in the A/P image area 86 as illustrated at 

20 block 150. If so, computer 40 advances to block 476 of Fig. 16. If not, computer 40 
then determines whether the Sagittal Image area was selected by positioning the cursor in 
the sagittal image area 62 on monitor 48 and clicking. If so, computer 40 advances to 
block 506 of Fig. 17. if not, computer 40 advances to block 1S4. 

From block 154, computer 40 determines whether the robot control area 

25 54 or 106 was selected by moving the cursor and clicking in the Robot Control area 84 on 
monitor 48 or the Robot Control area 106 on monitor SO. If the Robot Control was 
selected, computer 40 advances block 536 of Fig. 18. If the Robot Control was not 
selected, computer 40 advances to block 158 to determine whether the "Q" key was 
pressed indicating the operator desires to quit the main program. If the "Q** button was 

30 pressed, then computer 40 firees all allocated memory as illustrated at block 160 and ends 
the nuiin program as illustrated at block 162. If the "Q" button was not pressed at block 
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158, computer 40 advances back to block 122, waiting for a another button or key to be 
pressed. 

The various functions performed by the system of the present invention 
will be described in detail. If the Acquire A/P Image button 94 or the Fl key is pressed 
5 the, computer 40 advances to block 166 of Fig. 4. Computer 40 then determines whether 
the image acquisition card is in a passthrough mode at block 168. Button 94 and the Fl 
key are toggle buttons. When the button 94 or the Fl key is initially pressed, the card is 
in passthrough mode and images from the C*arm 12 are transmitted directly to the 
monitor 50. Whatever image is being taken by the C-arm is seen on the monitor 50 in the 

10 A/P image area 86. Therefore, if the card is not in the pass*through mode at block 168, 
pressing button 94 or the Fl key sets the pass-through mode at block 170. Computer 40 
then returns to wait for the next command as illustrated at block 172. When the button 
94 or the Fl key is pressed again after the image acquisition card within the computer 40 
is in pass-through mode, it freezes the live image and captures the A/P image as illustrated 

15 at block 174. This captured image is then displayed on monitor 50 as illustrated at block 
176. Computer 40 then disables and dims buttons Fl 1, F12 and F5, and enables and 
brightens button 96 and key F3 as illustrated at block 178. In other words, after the A/P 
image has been captured, computer 40 allows the operator to have the option to select the 
A/P fiducials through button 96 or key F3. 

2 0 Computer 40 then assigns a NULL tool as illustrated at block 1 80. The 

NULL tool of the robot is the three-dimensional location of end flange 22 of robot 18. In 
other words, the end flange 22 establishes a three-dimensional position for the robot, 
without depending on the particular surgical instrumentation which may be attached to 
the end flange 22. Computer 40 determines whether the NULL tool was property 

25 assigned at block 182. If not, computer 40 generates an error message "Tool Not 

Assigned!** as illustrated at block 184. Computer 40 then waits for the next command as 
illustrated at block 186. If the NULL tool is assigned properly at block 1 82, computer 40 
gets the current position of the end flange from the robot controller 53 as illustrated at 
block 188. Computer 40 then determines whether the sagittal image is displayed on 

30 monitor 48 as illustrated at block 190. If not, computer 40 sends a message of ''Acquire 
Sagittal Image" as illustrated at block 192, and then returns to wait for the next command 
at block 194. If the sagittal image is displayed at block 190, computer 40 sends the 
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message "Select the Fiducials" as illustrated at block 196. Computer 40 then returns to 
wait for the next command at block 198. 

If the Acquire Sagittal Image button 72 or the F2 key is pressed^ computer 
40 advances to block 200 of Fig. 5. Computer 40 then determines whether the image 
5 acquisition card is in a pass-through mode at block 202. Button 72 and the F2 key are 
toggle buttons. If the card is not in the pass-through mode at block 202, pressing button 
72 or the F2 key sets the pass-through mode at block 204. Computer 40 then returns to 
wait for the next command as illustrated at block 206. When the button 72 or the F2 key 
is pressed again after the image acquisition card within the computer 40 is in pass-through 

10 mode, it freezes the live image and captures the sagittal image as illustrated at block 208. 
This captured image is then displayed on monitor 48 as illustrated at block 210. 
Computer 40 then disables and dims buttons Fl 1. F12 and F6. and enables and brightens 
button 74 and key F3 as illustrated at block 212. In other words, after the sagittal image 
has been captured, computer 40 allows the operator to have the option to selea the 

15 sagittal fiducials through button 74 or key F4. 

Computer 40 then assigns a NULL tool as illustrated at block 214. 
Computer 40 determines whether the NULL tool was properly assigned at block 216. If 
not, computer 40 generates an error message "Tool Not Assigned!" as illustrated at block 
218, Computer 40 then waits for the next command as illustrated at block 220. If the 

20 NULL tool is assigned properly at block 216, computer 40 gets the current position of the 
end flange 22 from the robot controller 53 as illustrated at block 222. Computer 40 then 
determines whether the A/P image is displayed on monitor 50 as illustrated at block 224. 
If not, computer 40 sends a message of ''Acquire A/P Image" as illustrated at block 226, 
and then returns to wait for the next command at block 228. If the A/P image is displayed 

25 at block 224, computer 40 sends the message "Select the Fiducials** as illustrated at block 
230. Computer 40 then returns to wait for the next conunand at block 232. 

If the Select A/P Fiducials button 96 or the F3 key button is pressed, 
computer 40 advances to block 234 of Fig. 6. Computer 40 first determines whether the 
A/P image is displayed on monitor SO as illustrated at block 236. If not, computer 40 

3 0 generates an error message. "Acquire A/P Image" as illustrated at block 238. Computer 
40 then returns to wait for the next command as illustrated at block 240. 
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If the A/P image is displayed at block 236, computer 40 displays a square 
cursor on the display screen of monitor SO as illustrated at block 242. Computer 40 then 
resets the number of located fiducials to zero as illustrated at block 244. Next, computer 
40 waits for the uackball button to be clicked by the operator as illustrated as block 246. 
5 Once the trackball button is clicked over a fiducial shadow, computer 40 generates a beep 
as illustrated at block 248. Computer 40 then performs edge detection around the 
selected mouse cursor coordinate as illustrated at block 250. Such edge detection is 
performed using a gradient base method developed by John Canny and described in the 
article referenced in Section [1] of the attached Appendix. Such article is hereby 

10 incorporated by reference and made a part of this detailed description. 

Computer 40 then determines whether at least 3 edge pixels were found 
during the edge detection step as illustrated at block 2S2. If not, computer 40 generates 
an error message of "Try Again Closer to the Fiducial" as illustrated at block 254. 
Computer 40 then returns to block 246 to wait for the mouse button to be clicked again. 

15 If at least three edge pixels were found at block 252, computer 40 maps the edge pixels to 
their calibrated image coordinates using equation [13] from the attached Appendix as 
illustrated at block 256. 

Computer 40 then finds the center of the fiducial shadow generated by the 
fiducials 26 using the calibrated edge pixels as set forth in equation [14] of the Appendix. 

20 This step is illustrated at block 258. Computer 40 then advances to block 262 of Fig. 6. 
From block 262, computer 40 draws a circle around the center of the fiducial shadow. 

Computer 40 then determines whether all eight of the fiducials 26 have 
been located in the A/P image as illustrated at block 264. If not, computer 40 returns to 
block 246 of Fig. 6 and then waits for the mouse button to be clicked again over a 

25 different fiducial shadow. 

If all eight fiducials have been located at block 264, computer 40 then 
saves the established inuige coordinates of all the fidudals in the computer memory as 
illustrated at block 268. Computer 40 then enables and brightens the Register A/P Image 
Button 98 and FS key as illustrated at block 270. Computer 40 then transmits the 

3 0 message "^Register A/P Image" as illustrated at block 272. 
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Nexl, computer 40 aulomaiically advances to location ENTRY 1 of Fig. 8 
as illustrated at Block 274. Computer 40 does not wait for an operator to press a button 
to move to location ENTRY 1 of Fig. 8. 

If the Select Sagittal Fiducials or the F4 key button is pressed, computer 
5 40 advances to block 276 of Fig. 7. Computer 40 first determines whether the sagittal 
image is displayed on monitor 48 as illustrated at block 278. If not, computer 40 
generates an error message, ''Acquire Sagittal Image** as illustrated at block 280. 
Computer 40 then returns to watt for the next command as illustrated at block 282. 

If the sagittal image is displayed at block 278, computer 40 displays a 

10 square cursor on the display screen of monitor 48 as illustrated at block 290. Computer 
40 then resets the number of located fiducials to zero as illustrated at block 292. Next, 
computer 40 waits for the trackball button to be clicked by the operator as illustrated as 
block 294. Once the trackball button is clicked, computer 40 generates a beep as 
illustrated at block 296. Computer 40 then performs edge detection around the selected 

15 trackball cursor coordinate as illustrated at block 298. Such edge deteaion is performed 
using a gradient base method developed by John Canny and described in the article 
referenced in Seaion [1] of the attached Appendix. 

Computer 40 then determines whether at least 3 edge pixels were found 
during the edge detection step as illustrated at block 300. If not, computer 40 generates 

20 an error message of "Try Again Closer to the Fiducial** as illustrated at block 302. 
Computer 40 then returns to block 294 to wait for the trackball button to be clicked 
again. If at least three edge pixels were found at block 300, computer 40 maps the edge 
pixels to their calibrated image coordinates using equation [13] from the attached 
Appendix as illustrated at block 304. 

25 Computer 40 then finds the center of the fiducial shadow generated by the 

fiducials 26 using the calibrated edge pixels as set forth in equation [14] of the Appendix. 
This step is illustrated at block 306. Computer 40 then advances to block 310. From 
block 310, computer 40 draws a circle around the center of the fiducial shadow. 
Computer 40 then determines whether all eight of the fiducials 26 have been located in the 

3 0 sagittal image as illustrated at block 312. If not, computer 40 returns to block 294 and 
then waits for the trackball button to be clicked again. 
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If all eight fiducials have been located at block 312, computer 40 then 
saves the established image coordinates of all the fiducials in the computer memory as 
illustrated at block 316. Computer 40 then enables and brightens the Register sagittal 
Image Button 76 and the F6 key as illustrated at block 3 1 8. Computer 40 then transmits 
a message of "Register Sagittal Image" as illustrated at block 320. 

Next, computer 40 automatically advances to location ENTRY2 of Fig. 9 
as illustrated at block 322. Computer 40 does not wait for an operator to press a button 
to move to location ENTRY2 of Fig. 9, 

If the Register A/P Image buiion 98 or the F5 key was pressed, computer 
40 advances to block 324 of Fig. 8. Computer 40 first determines whether all of the A/P 
fiducials have been found as illustrated at block 326. If not, computer 40 generates an 
error message of "HavenH Selected All the Fiducials" as illustrated at block 328. 
Computer 40 then returns to wait for the next command as illustrated at block 330. 

If all the A/P fiducials have been found at block 326, computer 40 
advances to block 332. As discussed above, computer 40 also automatically advances to 
block 332 fi-om block 274 of Fig. 6 after all the fiducials have been selected. 

In block 332, computer 40 first recalls all the two-dimensional coordinates 
of the A/P fiducial centers. Next, the computer 40 reads in data fi-om a file of the 
three-dimensional coordinates of the center of the fiducials 26 as illustrated at block 334. 
The three-dimensional coordinates of the fiducials 26 are obtained using a Coordinate 
Measurement Machine (CMM). Therefore, this data provides information related to the 
actual location of the fiducials 26. Typically, these CMMed coordinates are obtained 
from the manufacturer of the registration artifact 24. 

Next, computer 40 optimizes the parameters of a geometric model which 
projects three dimensional coordinates into corresponding image points. The optimized 
model is encapsulated in a registration matrix as set forth in section [3]. Optimization is 
performed by minimizing (in a least squares sense) the deviation between the model's 
projections of the three-dimensional coordinates read at block 334, and the 
two-dimensional coordinates read at block 332. The Levenberg-Marquardt method is 
used for optimization, as described in equation [2] of the attached Appendix and as 
illustrated at block 336 . Computer 40 then constnicts a registration matrix as set forth in 
section [3] of the attached Appendix. This step is illustrated at block 338. 
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Computer 40 next determines, whether the sagiual image has been 
registered as illustrated at block 340. If not, computer 40 generates a message of 
"Perform Sagittal Registration" as illustrated at block 342. Computer 40 then returns to 
wait for the next command as illustrated at block 344. 
5 If the sagittal image has been registered at block 340, computer 40 

generates a display message of "Pick Entry Point" as illustrated at block 346, Computer 
40 then returns to wait for the next command as illustrated at block 348. 

If the Register sagittal Image button 76 or the F6 key have been pressed, 
computer 40 advances to block 350 of Fig. 9. Computer 40 first determines whether all 
10 of the sagittal fiducials have been found as illustrated at block 352. If not, computer 40 
generates an error message of "Haven*t Selected All the Fiducials" as illustrated at block 
354. Computer 40 then returns to wait for the next command as illustrated at block 356. 

If all the sagittal fiducials have been found at block 352, computer 40 
advances to block 358. As discussed above, computer 40 also automatically advances to 
15 block 358 from block 322 of Fig. 7 after all the fiducials have been selected. 

In block 358, computer 40 first recalls all the two-dimensional coordinates 
of the sagittal fiducial centers. Next, the computer 40 reads in data from a file of the 
three-dimensional coordinates of the center of the fiducials 26 as illustrated at block 360. 
The coordinates of the fiducials 26 are obtained using a Coordinate Measurement 
20 Machine (CMM). Therefore, this data provides information related to the actual location 
of the fiducials 26. Typically, these coordinates are obtained from the manufacturer of the 
registration artifiict 24. 

Next, computer 40 optimizes the fit between the three-dimensional 
coordinates read at block 360 and the two-dimensional coordinates read at block 358 
25 using the Levenberg-Marquardt method described in equation [2] of the attached 

Appendix as illustrated at block 362. Computer 40 then constructs a registration matrix 
as set forth in section [4] of the attached Appendix. This step is illustrated at block 364. 

Computer 40 next determines whether the A/P image has been registered 
as illustrated at block 366. If not, computer 40 generates a message of "Perform A/P 
30 Registration" as illustrated at block 368. Computer 40 then returns to wait for the next 
command as illustrated at block 370. 
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If the A/P image has been registered at block 366, computer 40 generates 
a message of Tick Entry Point" as illustrated at block 372. Computer 40 then returns to 
wait for the next command as illustrated at block 374. 

If the transverse angle bunon 100 or the F7 key is pressed, computer 40 
advances to block 376 of Fig. 10. The transverse angle is the angle determined by using 
the right hand nile about the X axis 34 of Fig. 1 . To adjust the transverse angle, the 
operator places the cursor in the Entry Field button 101 of Fig. 3a as illustrated at block 
378 of Fig. 10. The operator then enters a numeric value for the transverse angle as 
illustrated at block 380. Computer 40 then reads the new transverse angle, and updates 
the orientation of the virtual guidewire using the equations set forth in section [6] of the 
attached Appendix. This step is illustrated at block 382. Next, computer 40 redraws the 
virtual guidewire projection 92 in the A/P image area 86 and 68 in the sagittal image area 
62 based on the new transverse angle using the equation set forth in section [7] of the 
atuched Appendix as illustrated at block 384. Computer 40 then returns to wait for the 
next command as illustrated at block 386. 

If the Screw Length button 80 or the F8 key was pressed, computer 40 
advances to block 388 of Fig. II. The cursor is then placed on the entry field 81 of Fig. 
3b as illustrated at block 390. The operator then enters the numeric value for the new 
screw length as illustrated at block 392. Computer 40 reads the new screw length, and 
updates the length of the virtual guidewire using the equations set forth in section [1 1] of 
the Appendix. This step is illustrated at block 394. Next, computer 40 redraws the 
projected guidewire 92 in the A/P image area 86 and the projected guidewire 68 in the 
sagittal image area 62 using the equations set forth in section [7] of the Appendix, These 
steps are illustrated at block 396. Next, computer 40 returns to wait for the next 
command as illustrated at block 398. 

If the Sagittal Angle button 78 or the F 10 key is pressed, computer 40 
advances to block 400 of Fig. 13 to adjust the sagittal angle. The sagittal angle is the 
angle about the Y-axis 36 of Fig. 1 using the right hand rule. 

The cursor is placed in an entry field 79 of Fig. 3b as illustrated at block 
402. Next, the operator enters a numeric value for the sagittal angle as illustrated at 
block 404. Computer 40 then reads the value of the new sagittal angle, and updates the 
orientation of the virtual guidewire using the equations set forth in section [10] of the 
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Appendix. These steps are illustrated at block 406. Next, computer 40 redraws the 
projected guidewire 92 in the A/P image area 86 and the projected guidewire 68 in the 
sagittal image area 62 using the equations set forth in section [7] of the Appendix. These 
steps are illustrated at block 408. The computer 40 then returns to wait for the next 
5 instruction as illustrated at block 4 1 0. 

If the Approach Angle button 102 or the F9 key was pressed, computer 40 
advances to block 412 of Fig. 12 The approach angle is the angle taken about the Z*axis 
38 of Fig. 1 uang the right hand nile. 

The cursor is placed in the entry field 103 of Fig. 3a as illustrated at block 
10 414. The operator then enters a numeric value for the new approach angle as illustrated 
at block 416. The computer 40 then reads the new approach angle as illustrated at block 
418. Computer 40 then returns to wait for the next command as illustrated at block 420. 

In order to plan a linear trajectory in space, only two angles are needed, 
for this particular procedure the transverse angle and the sagittal angle are used. The 
15 approach angle permits the surgeon to control movement of the robot. In other words, 
the approach angle is not used with the planning of the trajectory. 

If the Move Robot button 104, or the Fl 1 key are pressed, computer 40 
advances to block 422 of Fig. 14. Computer 40 first recalls the approach angle from 
memory as illustrated at block 424. Next, computer 40 recalls the sagittal angle, the 
2 0 transverse angle and the three-dimensional coordinates of the top point of the virtual 

guidewire as illustrated at block 426. Next, computer 40 calculates the plaxuied position 
and orientation using the equations in section [12] of the Appendix. This step is set forth 
at block 428. Next, computer 40 reads in data from a file related to the specific surgical 
end-effector being used for the surgical procedure as illustrated at block 430. This data 

2 5 includes the three-dimensional coordinates from the Coordinate Measurement Machine 

(CMM). 

Computer 40 determines whether the surgical end-effector is properly 
assigned at block 434. If not, computer 40 generates an error message of ''Surgical 
end-effector Not Assigned" as illustrated at block 436. Computer 40 then returns to wait 

3 0 for the next conunand as illustrated at block 438. 

If the surgical end-effector is properly assigned at block 434, computer 
sends a conunand through serial communication port SO to the robot controller S3 to 
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move the robot to the planned position and orientation as illustrated at block 440. 
Computer 40 assigns the "NULL" end-effector as illustrated at block 442. Computer 40 
detennines whether the NULL end-effeaor was properly assigned at block 444. If not, 
computer 40 generates an error message of "NULL end-effector Not Assigned" as 
illustrated at block 446. Computer 40 then returns to wait for the next command at block 
448. If the NULL end-effector is properly assigned at block 444, computer 40 returns to 
wait for the next conunand as illustrated at block 450. 

If the Move Robot Along Axis button 82 of Fig. 3b is selected, computer 
40 advances to block 452 of Fig. 15. The computer 40 has already moved the robot to 
the proper orientation during the steps of Fig. 20. Therefore, the steps of Fig. 21 are 
designed to move the robot along the tool guide axis defmed by the tool guide 28 of Fig. 
2. The tool guide axis typically moves toward and away from the body on the table 14 
along the tool guide axis. Computer 40 daermines whether a thread entitled "Move 
Robot Axis" has been dispatched at block 454. This thread program luns by itself until it 
is stopped. If the program is not started at block 454, computer 40 starts this program as 
illustrated at block 456. Computer 40 then returns to wait for additional instructions at 
block 458. If the thread program has started at block 454, then computer detennines 
whether the Page Up button has been pressed at block 460. If not, computer 40 
determines whether the Page Down button has been pressed at block 462. If not. 
computer 40 returns to block 464 to wait for the next command. 

If the Page Up button was pressed at block 460, computer 40 determines 
whether the Page Up button is still being pressed at block 466. If not, computer 40 
returns to wait for the next command as illustrated at block 468. If the Page Up button is 
still bang pressed at block 466, computer 40 sends a VAL command from 
communication port 50 to robot controller 53 to move the robot in the positive tool guide 
axis direction as illustrated at block 470. The positive tool guide axis direction is up away 
from the patient. Computer 40 then returns to block 466. 

If the Page Down button has been pressed at block 462, computer 40 
detennines whether the Page Down button is still being pressed at block 472. If not, 
computer 40 returns at block 468 to wait for the next command. If the Page Down 
button is still being pressed at block 472, computer 40 sends a VAL command to move 
the robot in the negative tool guide axis direction as illustrated at block 474. The 
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negative tool guide axis direction is down toward the patient. Computer 40 then returns 
to block 472. 

In other words, the control steps of Fig. 15 permit the operator to move 
the robot along its tool guide axis. Once the robot is moving in either the positive or 
5 negative direction, it keeps moving until the Page Up or Page Down are released. The 
entire robot moves in order to maintain the end-effector 24 and the tool guide 28 in the 
same orientation along the planned axis. In other words, the end-effector 24 of robot 1 8 
may be maintained in an orientation that is 45" relative to Z-axis 38 of Fig. 1 . VAL is the 
program control language for the PUMA-560 controller 53. It is understood that other 
10 robots, controllers, and program languages may be used in accordance with the present 
invention. 

If a cursor is over the A/P image area 86 of Fig. 3a, computer 40 advances 
to block 476 of Fig. 16. Computer 40 waits for the trackball to be clicked in the A/P 
image area 86 as illustrated at block 478. Once the trackball has been clicked at block 

15 478, computer 40 determines whether both the A/P image and the sagittal image have 
been registered as illustrated at block 480. If not, computer 40 does nothing and returns 
to block 482 to wait for the next command. 

If the A/P and the sagittal images have been registered at block 480, 
computer 40 determines whether the projected guidewire is drawn as illustrated at block 

20 484. If not, computer 40 assumes that the operator intends to draw the projected 

guidewire. Therefore, the computer 40 draws a cross hair at trackball coordinate (IJy) 
as illustrated at block 486. Next, computer 40 draws a curve representing the line of site 
on the sagittal image using the equations of section [5] of the attached Appendix as 
illustrated at block 488. A curve is drawn representing the line of site due to the 

25 distortion in the images. If you take an x-ray of a straight . line, its image will be a curve 
due to the distortions inherent in the fluoroscope's image intensifier. This is why a curve 
must be drawn to represent the line of sight. Once the line of sight indicator 70 is drawn 
on the sagittal image area 62 of Fig. 3b, computer 40 returns to wait for the next 
command as illustrated at block 490. 

30 If the projected guidewire is already drawn at block 484, computer 40 

determines whether the trackball coordinates are within five pixels from the top point 88 
in the A/P image area 86. This step is illustrated at block 492. If the cursor coordinates 
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are within five pixels from the top point 88, computer 40 erases the projected guidewire 
as illustrated at block 494 and returns to wait for the next command as illustrated at block 
496. 

If the trackball cursor coordinates are not within five pixels from the top 
point 88 at block 492, computer 40 determines whether the trackball coordinates arc 
within five pixels of the bottom point 90 as illustrated at block 498. If not, computer 40 
returns to wait for the next command as illustrated at block 490. If the trackball cursor 
coordinates arc within five pixels from the bottom point 90 at block 498, computer 40 
determines whether the trackball has been clicked again as illustrated at block 500. If so. 
computer 40 returns to block 490 to wait for the next command. If not, computer 40 
updates the transverse or sagittal angle as illustrated at block 502 based on movement of 
the trackball. The transverse angle value is incremented if the trackball is being moved 
up. The transverse angle value is decreased ifthe trackball is moving down. The sagittal 
angle value is incremented if the trackball is being moved right. The sagittal angle value is 
decreased ifthe trackball is moving left. The incrementing factor is 0, T per pixel. The 
equations for this step are set forth in section [6] of the Appendbc. 

After the transverse and/or sagittal angle have been updated at block S02, 
computer 40 redraws the projected guidewire 92 in the A/P image area 86 and the 
projected guidewire 68 in the sagittal image area 62 using the equations in section [7] of 
the attached Appendix. These steps are illustrated at block 504. Computer 40 then 
returns to block 500. 

Ifthe cursor is over the sagittal image area 62 of Fig. 3b, computer 40 
advances to block 506 of Fig. 17. Computer 40 determines whether the line of sight has 
been drawn at block 508. If not, computer 40 returns to wait for the next command at 
block 510. Ifthe line of sight has been drawn at block 508, computer 40 draws the 
projected guidewire 92 in the A/P image area 86 and the projected guidewire 68 in the 
sagittal image area 62 using the equations in section [8] of the Appendix. These steps are 
illustrated at block 512. Computer 40 also checks ifthe robot has been initialized at block 

513, if it has then computer 40 enables and brightens buttons "Move Robot** 104, and 
"Move Along Drill Axis" 82. and keys Fl 1, and F12 at block 513.5. Next, computer 40 
waits for the track ball in the sagittal image area 62 to be clicked as illustrated at block 

514, If robot has not been initialized then computer 40 waits for the track ball in the 
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sagittal image area 62 to be clicked as illustrated at block 514. Next, computer 40 
determines whether the trackball cursor coordinates are within five pixels from the top 
point 64 as illustrated at block 516. If not, computer 40 determines whether the trackball 
coordinates are within five pixels of the bottom point 66 as illustrated at block 518. If 
5 not, computer 40 returns at block 520 to wait for the next command. 

If the trackball coordinates are within five pixels of the top point 64 at 
block 516, computer 40 determines whether the trackball has been clicked again at block 
522, Ifso, computer 40 returns at block 524 to wait for the next conrunand. If not, 
computer 40 updates the position of the virtual guidewire 68 by moving it along the line 

10 of sight in the same direction as the trackball movements. The incrementing ratio is 0. 1 
mm/pixel. This step is illustrated at block 526. The computer uses the equations set forth 
in section [9] of the Appendix to update the virtual guidewire position. Computer 40 then 
redraws the projected guidewire 68 in the sagittal image area 62 and also redraws the 
projected guidewire 92 in the A/P image area 86 as illustrated at block 528 by using the 

15 equations set forth in Section [7] of the Appendix. Computer 40 then returns back to 
block 522. 

If the trackball coordinates are within five pixels from the bottom point 66 
at block 518, computer 40 determines whether the trackball has been clicked again at 
block 530. Ifso, computer 40 returns at block 524 to wait for the next command. If not, 

2 0 computer 40 assumes that the operator wants to adjust the position of bottom point 66. 
Therefore, computer 40 updates the sagittal and/or transverse angle as illustrated at block 
532 based on movement of the trackball. The transverse angle value is incremented if the 
trackball is being moved up. The transverse angle value is decreased if the trackball is 
moving down. The sagittal angle value is incremented if the trackball is being moved to 

25 the right. The sa^ttal angle value is decreased ifthe trackball is moving to the left. The 
incrementing ratio is 0. 1 ° /pixel. Computer 40 uses the equations of section [10] of the 
Appendix for these steps as illustrated at block 532. Next computer 40 redraws the 
projected guidewire 68 in the sagittal image area 62 and the projected guidewire 92 in the 
A/P image area 86 as illustrated at block 534 using the equations set forth in Section [7] 

30 of the Appendix. Computer 40 then returns to block 530. 

If the Robot Control areas 84 of Fig. 3a-b is selected, computer 40 
advances to block 536 of Fig. 18. Computer 40 then displays a menu giving the user 
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options at block 538. The first option is a ''Initialize Robot" option. Computer 40 
detemrunes whether the Initialize Robot menu item was selected at block S40. If so, 
computer 40 opens the serial communication port 52 for communication with the robot 
controller 53 as illustrated at block 542. Computer 40 sends the VAL program language 
commands required to initialize the robot controller 53 as illustrated at block 544. 
Computer 40 determines whether VAL was initialized properly at block 546. If VAL was 
not initialized properly then the computer 40 sends message VAL not initialized 53 as 
illustrated at block 548. Computer 40 then returns at block 550. 

If VAL was properly initialized at block 546. computer 40 transmits 
preestablished HOME and START positions to the robot controller 53 as illustrated at 
block 552. The HOME and START position are two positions in the work space of the 
robot. In addition, computer 40 initializes the preestablished NULL end-effector and 
SURGICAL end-effector as illustrated at block 554. In other words, computer 40 sends 
specifications to the precise configurations of the specific surgical instrument that is going 
to be used. Therefore, the controller 53 is programmed to move the robot to these 
positions. During operation, computer 40 can instruct the controller 53 to move to the 
particular HOME or START positions. In addition, controller 53 will recognize 
instructions for the particular surgical end-effector which was initialized during step 554. 
Next, the robot speed is set to a very slow speed as illustrated at block 556. For example, 
the robot speed is set to a speed of 5 out of 256. Next, the computer 40 checks if the 
virtual guidewire has been planned, if it has then it enables and brightens buttons "Move 
Robot" 104 and "Move Robot Along Tool Axis" 82 and keys Fl 1, F12, as illustrated in 
block 557.5. Computer 40 then r^ums to wait for the next instruction as illustrated at 
block 559. 

If the virtual guidewire has not been planned, computer 40 then returns to 
wait for the next instruction as illustrated at block 558. 

If an option entitled "Move to a Predefined Location" was selected firom 
the pop-up menu 538 and if the robot was already initialized as illustrated at block 560, 
then computer 40 displays a dialog box with options to move the robot to the predefined 
locations as illustrated at block 562. In other words, a dialog box with the options to 
move the robot to the HOME position or the START position are displayed. The 
operator can select one of these options at block 562. Computer 40 then sends a VAL 
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command to controller 53 to move the robot 18 to the specified location as illustrated at 
block 564. Computer 40 then returns at block 568 to wait for the next command. 

If computer 40 determines that the option "Assigned Predefmed Tool'* was 
selected from the menu 538 and if the robot has already been initialized as illustrated at 
block 570, then computer 40 displays a dialog box with options to assign the predefined 
tools established during the initialization step at block 554. This step is illustrated at 
block 574. In other words, computer 40 displays a dialog box for assigning cither the 
NULL end-effector or the SURGICAL end-effector at block 574. Once the desired tool 
is selected, computer 40 transmits to VAL the command to assign the specified 
end-eflfector to controller 53 as illustrated at block 576. Computer 40 then returns to 
wait for the next command at block 578. If the assigned predefined end*effector item was 
not selected or the robot was not initialized at block 570, computer 40 returns at block 
572 to wait for the next command. 

Although the invention has been described in detail with reference to a 
certain preferred embodiment, variations and modifications exist within the scope and 
spirit of the present invention as described and defined in the following claims. 
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Notation used throughn nt the flnwrhartg 

WCS 

CCS 

(x.y) 
(«.v) 
a 
fi 

r 

Subscripts: 

w = WCS 
c = CCS 

t«top 
b = bonoxn 

a = A/P 
s = Sagittal 



- 29 • 

APPENDIX 
(Page 1 of 6) 



World Coordinate System 

C-aim Coordinate System 

Used for 3D coordinates in WCS and the CCS. 

Used for calibrated image coordinates. 

Used for real image coordinates. 

Sagittal Angle. 

Transverse Angle. 

Approach Angle. 

Specifics die coordinate system. Only used with 3D 
coodinate systems. 

Specifies a point on the virtual guidewire. 
Specifies to what image the information pertains to. 



[I] J. Canny; "A Computational Approach to Edge Detection"; IEEE Transactions on Pattern Analysis Machine 
Intelligence; Vol 8, Nov. 1986, pp. 679-698. 

[2] Mathematics involved in perforaiing flie Levcnbcrg-Marquardt optimiziion method: 



cosmos ^ cos^in£^in{^-sin^os{f^ cos^inftos^+ si 
sin^os^ sin^inftin 4- cosmos ^ sin^in6bost^ 
-sin^ cosftiny' cosftos^f^ 



+ sin^inf^ I 
. cos^in^ 

»S(f^ J 



ind 



and 

V, 



i-o 



where - [x,>. 2], are the 3D coordinates of the fiducials. (u, v) arc the 2D coorduiates of the center of the 
fiducials, and a " e, ^, t^, g are the six parameters that deflne a six degree-of-fteedompose. 



13] 



Once the fit has been performed 1 construct the homogeneous transformation matrix Hat corresponds to the 
optimized parameters (a » ft ^. 1^, t^, g) as follows: 
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REG. 



cos^cos9 cos^sindsinit' - sin^cosM^ 

sin^cosB sitt^sin8sin\|/ cos^osv 

-sinG cosBsiou/ 

0 0 



cos^sinScosH/ siD4)sin\|/ px 

sin^sinBcosv}; • cos^inv^ py 

cosBcosHT pz 

0 1 



[4] Once the fit has been performed I contruct the homogeneous transformation matrix diat conesponds to the 
optimized parameters (a « [A A 'y 'J) &s follows: 



R£Gb< 



cos^cosG cos^sin6$in\); - sin^cosv 

sin^osO sin^sindsiny; -t- cos^s\|; 

-sinS cosBsiny 

0 0 



cos^in8cos\(^ -t* sin^sinv px 

sin^sinBcosH/ - cos^siny py 

C056C0$V|/ pz 

0 1 



[5] The line of sight is calculated in the following way: 

The line of sight is bound by (0, 0, 0) and fy^ v^,f) in the CCS. 
Note: (uci Vc) is the cahl)rated equivalent of (v» v). See [13] 





LSx^2 






0 


LSy^x 


LSz^2 




V. 
/ 


0 
0 


L 1 


1 - 




L I 


1 . 



>ol yaZ 
1 1 J 



'[REGa] 



LSx«] LSx«2 

LSy»i LSy«2 

LSzwi LSz»2 

L 1 1 



U2 



Due to the inherent distortion in the fluoroscopic images the line of sight is drawn as a curve image. This is done 
by urt'calibrating 50 points on the line bound by (tij, V|) and (1/21 vj) ^ >n 115] and drawing a polyline through 
^em. 



[6] Recall that the virtual guidewire is a 3D object bound by (GU« 0«„ Owi) and (Ow^ 0»^, •screwlength^). 
^ - ^ + 0.1 • (# pixels moved by the trackball) 









0., 


0.b 




yy^ 


-[T'(a,A«.0'.te)l 


Owi 


0.b 

•screwlength^ 


L 1 


I J 




L 1 


1 



[7] With ( Kxm, Vywi. yzwt) and ( Vx^^ Vy^, Vz^) the virtual guidewire's projection is drawn on both the A/P and 
sagittal images using the following equations: 



wo 97/42898 



• 31 . 



PCT/US97/D8128 



>ot 
. 1 


yah 
•cab 

1 J 


-[REG*] 


L I 


yy^ 
yt^ 
1 J 


2esi 
L 1 


X<tb 

yob 

1 J 


-iREGs] 


' yx^ 
yy^ 
yt^ 

. 1 


yy^k 
1 J 
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Due to the distortion in fluoroscopic images the projected guidewire is drawn as a curve. This is done byi/u- 
calibrating 20 points on tbc line bound by {u^, v„) and («^, v^) as in [15] and drawing a polyline through them 
on the A/P image and similarly for the Sagittal image using (i^i Vn) and {u^ v^). 



[8] To draw the vinual guidewirc's projection, two points (0, 0, 0) and (0, 0. -screwlength), in the WCS arc 

transformed so that the top point (0, 0, 0} lies on the line of sight. The virtual guidewire is initiaUy set to 30mm. 
The projected guidewire is drawn using the followix^ math: 

initially: 
depth 

screwlength » 30nmi 

(tt, ty, tz) is constrained to lie on the line of sight bound by (LSx.,, LSy^u and (LSx^j. LSy.j, i-fo*a), thus 

0r « LSx^i . depth*(LSx^2 • LSx^i) 
ty = ISy^x - depth*(LSy^2 - ISywt) 
tz = ISz^x - d^th*(LSz^2 • LSz^i) 



yy^ Vy^b 

Vz^ Vz^ 

1 ] J 



{V{a,fi.tx.ty,ti)] 





0^ 








sarevAength 


L 1 


1 



' T is composed of the following transformations: 
T « Trans(a. ty, tz) Rot(y, a) Rot(x, 
or ^ 

cosa sinasio^ sinacos^ fir 
0 cos^ -sin/? ty 

-sina cososin^ cosocos)? tz . 



[T(a,Att.(v,fr)] 
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in order to draw the projected guidewire on the images, the points (Kt«. Vy^,Vz^ and (Kx^, Vy^ Vz^) are 
used in conjunction [7]. 

[9] Recall diat the virtual guidewire is a 3D object bound by (O,, 0^ 0,J and (0^. 0.^. screwlength^). 

depth = depth + 0. 1 • (# pixels moved by the trackball) 

fic «■ LStwi - depth*{LSx^i • L&,|) 
ty - LSy^y . depth*(LSy^j . LSy„) 
tt = - depth*(LSz^i . LSr„) 



L 1 1 



^\l\a,fi,tx,ty,t£i\ 



Ov, -screwlength^ 
1 1 



[10] Recall that the virtual guidewire is a 3D object bound by (d,, 0«. 0«) and (0,^. 0^. screwlength^). 
a = a •<■ 0. 1 * (# pixels moved by the trackball) 



yy., yy^ 

L 1 1 



'^[l\a,p,tx,ty.ti)] 



Om -screwlength^ 
L 1 1 



(11] Recall tbat Oie virtual guidewire is a 3D object bound by (CU, 0«, 0„) and {0^ 0^, .scrwUngth^). 
'[V(a,p,tx,ty,tz)] 



Vx^ yx^ 

yy^ yy^ 

yz^ vz^ 
1 1 J 



0« Ort 

On 'SerewUHgth^ 
L 1 1 



[12] Given 



(Tool]' =(R0t(z, -90)1 [R0t(y. .90)1 
IPu„l^ = lR0t(y.a)l(R0t(x.P)lITc6il 
iAppro.chl*=JRot(z.T)l _ 

(FiralPunr- Pn, ? ? 

L Ph. J 

and using the following contraints I determine the remaining two vectors that would be completePTP]. 
Note: The first vector (N) is maintained from the (Plan) since it is the dirll guide axis: 



Contraints: 



i. These matrices^ of the following form: 



[mese matnces.are 
Nx Ox Ax I 
Ny Oy Ay 
Nz Oz Az J 
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1) FPj^FPAr + FPA.*-l 

2) A„FPa = 0 

3) FPhFPa«0 

p Anv(Av,>FP^„.FPm,>Aw,> 

An. (FPnx • An, - An.- FPwy) " An» 

(An. - FPn, - FPm, A^} 

(FPN, ANy-AN«- FPNy) 

FPa* = WD' + E^+I 
FPa.«D FPax 

FPAy-E FPA, 

FPo is detennined using 

FPo'^FPnxFPa 

Hence, 

rw. « , r Pn. FPo. FPa. 1 
[FinalPlan]" Pn, FPo, FP^y 
L Pn, FPo. FPa. J 

Since the PUMA 560 robot uses an Euler lepresentatSon for specifying an orientation, the inverse solution ofTFP] 
is determined in the following manner. 

Euler representation = Rot(z,« Rot(y. 6) Rot(2, y) dius from p ]. 

^ -arctan(/7>Ay,F/>Aj 

0 = arctan(/7>A, ' cos(fi + FP^y • 8in(rt. FP^ 

yf - arctan(-£PN« • sin(^ + FPn, • cos(^, .F/>o, • sin(^J) + FPoy • cos(^) 

Adding a PUMA specific offset to ^, and ^the final position and orientation is established 

Final pose - ((i^ + 90. 90, y^, ft, ty, tz) 



113] The calibrated coordinates (x,y) of the edge-pixels («, v) are determined using a quartic polynomial equation as 
follows: 

y = Co i/*v* + a^ + o, uW ... + fl2j uv + 02< 

the set of parameters a and ^, are previously determined using the image calibration program. 

[I4J The center of the fiducial shadow is found by fitting the equation of a circle to the edge-pixels using a pseudo- 
inverse approach: 
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or 

A-BP 

using pseudo inverse 

once P ts established the center of the fiducials (h, k) is detennined as follows: 




[ 1 5] The un-calibrated (distorted) coordinates (ii, v) conesponds to the calibrated coordinate (x, y) and is detennined 
using a quartic polynosiial equation as follows: 

V « flo « V + O I JCV + 02 V + . + ^23 JCy + 024 

ttie set of parameters a and 6, are previously detennined using a separate calibration progranL 



i Robot Manipulators! Mathematies Preyramminy md Control; Richard P. Paul; The MIT Press, Cambridge, 
Massachusetts and London, England, 1983. 
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CLAIMS: 

1 . A method for planning a stereotactic surgical procedure using a 
fluoroscope for generating images of the body, the method comprising the steps of 

5 placing adjacent to the body a registration artifact including a plurality of 

fiducials at known positions relative to a known coordinate frame of the artifact; 

displaying on a computer monitor an image taken of the patient's body and 
the registration artifact; 

receiving an input to identify two-dimensional coordinates of the fiducials 
10 of the registration artifact displayed on the image; and 

registering the image by creating a geometric model having parameters, 
said model projecting three-dimensional coordinates into image points, and numerically 
optimizing the parameters of the geometric model such that the projections of the known 
three-dimensional coordinates of the fiducials best fit the identified two-dimensional 
15 coordinates in the image. 

2. The method of claim 1, further comprising the steps of 
displaying a second image taken of the patient's body and the registration 

artifact but from an angle different from that of the first image; 

receiving an input to identify two-dimensional coordinates of the fiducials 
20 of the registration artifaa displayed on the second image; and 

registering the second image by creating a geometric model having 
parameters, said model projecting three-dimensional coordinates into image points, and 
numerically optimizing the parameters of the geometric model such that the projections of 
the known three-dimensional coordinates of the fiducials best fit the identified 
25 two-dimensional coordinates in the second image. 

3. The method of claim 2, further comprising the step of receiving a 
user input to select a point upon the first image, said point partially designating a virtual 
guidewire. 

4. The method of claim 3, further comprising the step of receiving an 
30 input specifying a position, a length, and angles of the virtual guidewire. 
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5. The method of claim 4, further comprising the step of drawing 
projected guidewire segments on the images, such that the projected guidewires are 
projections of the virtual guidewire onto the images. 

6. The method of claim 5, further comprising the steps of receiving a 
5 user input to move either end of the projected guidewire on either image, by revising the 

virtual guidewire of which the two projected guidewires are projections, and by redrawing 
the two projected guidewires on their respective inrmges in correspondance with the 
revised virtual guidewire. 

7. The method of claim S, further comprising the steps of receiving a 
10 user input to change the length of the virtual guidewire. and redrawing the two projected 

guidewires on their respective images in correspondance with the revised virtual 
guidewire. 

8. The method of claim S, further comprising the steps of receiving a 
user input to change the sagittal angle of the virtual guidewire, updating the orientation of 

15 the virtual guidewire based on the new sagittal angle, and redrawing the two projected 
guidewires on their respective images in correspondance with the revised virtual 
guidewire. 

9. The method of claim 5, further comprising the steps of receiving a 
user input to adjust the transverse angle of the virtual guidewire, updating the orientation 

20 of the virtual guidewire based on the new transverse angle, and redrawing the two 

projected guidewires on their respective images in correspondance with the reused virtual 
guidewire. 

10. The method of claim S, further comprising the steps of receiviiig a 
user input to adjust the coronal angle of the virtual guidewire, updating the orientation of 

25 the virtual guide\sdre based on the new coronal angle, and redrawing the two projected 
guidewires on their respective images in correspondance with the revised ^rtual 
guidewire. 

1 1 . The method of claim 5, further comprising the step of producing an 
output to adjust the coordinates of a tool guide such that its axis is brought into alignment 

3 0 with the virtual guidewire. 

12. The method of claim 1 1, further comprising the step of producing 
an output to adjust the coordinates of a tool guide such that the position of the guide 
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along hs axis is offset by a preselected distance from one endpoint of the virtual 
guidewire. 

13. The method of claim 1 1, further comprising the step of transmitting 
said coordinates to an automatic mechanical device. 
5 14. The method of claim 1 1, further comprising the step of displaying 

said coordinates with which a human operator may manually adjust a mechanical device. 

15. The method of claim 1 1, wherein the registration artifact includes a 

tool guide. 

16. The method of claim 2, further comprising the step of receiving an 
10 input to select a point upon the first image, said point partially designating a virtual 

targetpoint for a surgical instrument. 

17. The method of claim 16, further comprising the step of drawing a 
projected targetpoint both on the first image and another on the second image, such that 
the projected targetpoints are projections of a virtual targetpoint onto the images. 

18. The method of claim 17, further comprising the steps of receiving a 
user input to move the projected targetpoint on either image, by revising the virtual 
targetpoint of which the two projected targetpoints are projections, and by redrawing the 
two projected targetpoints on their respective images in correspondance with the revised 
virtual targetpoint. 

19. The method of claim 18, further comprising the step of producing 
an output to adjust the coordinates of a tool guide such that its axis intersects the virtual 
targetpoint. 

20. The method of claim 19, further comprising the step of producing 
an output to adjust the coordinates of a tool guide such that the position of the guide 
along Its axis is offset by a preselected distance from the virtual targetpoint. 

2 1 . The method of claim 1 9, further comprising the step of transmitting 
said coordinates to an automatic mechanical device. 

22. The method of daim 19, further comprising the step of displaying 
smd coordinates with which a human operator may manually adjust a mechanical device. 

23. The method of claim 19, wherein the registration artifact includes a 

tool guide. 
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24. The method of claim 1, further comprising the step of receiving an 
input to select a point upon the first image, said point partially designating a virtual 
guidewire representing a trajectory for the surgical instrument into the body. 

25. The method of claim 24, fiirther comprising the step of producing 
5 an output to adjust the coordinates of a tool guide such that its axis is brought into 

alignment with the virtual guidewire. 

26. The method of claim 2S, fiirther comprising the step of transmitting 
said coordinates to an automatic mechanical device. 

27. The method of claim 25, further comprising the step of displaying 
10 said coordinates with wluch a human operator may manually adjust a mechanical device. 

28. The method of claim 25, wherein the registration artifact includes a 

tool guide. 

29. An apparatus for planning a stereotactic surgical procedure using a 
fluoroscope for generating images of the body, the apparatus comprising: 

15 means for placing adjacent to the body a registration artifact including a 

plurality of fiducials; 

means for displaying an image taken of the body and the fiducials; 
means for identifying two«dimensional coordinates of the fiducials in an 

image; 

20 means for registering an image with respect to said fiducial arti&ct; 

means for receiving inputs to select and adjust a ^rtual guidewire or 
targetpoint, while the projections of said guidewire or targetpoint are displayed 
superimposed upon the image; and 

means for producing an output to adjust the coordinates of a tool guide. 
25 30. An apparatus for planning a stereotactic surgical procedure for a 

linear trajectory insertion of a surgical instrument into a body using a fluoroscope for 
generating images of the body, the apparatus comprising: 

a registration artifact located adjacent to the body, the registration artifact 
including a plurality of fidudals located at known three-dimensional coordinates relative a 
3 0 known coordinate frame; 

means for displaying at least one image taken of the body and the fiducials 
on at least one computer monitor; 
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means for identifying two-dimensional coordinates of the fiducials in each 

image; and 

means for numerically optimizing parameters of a geometric model, said 
model projecting three-dimensional coordinates into image points, such that the 
5 projections of the known three-dimensional coordinates of the fiducials best fit the 
identified two-dimensional coordinates in the image. 

3 1 . The apparatus of claim 30, further comprising a means for 
receiving user input to select a position, a length, and the angles of a virtual guidewire; 
and means for displaying a projected guidewire segment on each registered image 

10 representing the location of the virtual guidewire. 

32. The apparatus of claim 30, further comprising a tool guide, and 
means for producing an output to adjust the coordinates of the tool guide. 
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